Multi-agent reinforcement learning for Markov routing games: A new modeling paradigm for dynamic traffic assignment

This paper aims to develop a paradigm that models the learning behavior of intelligent agents (including but not limited autonomous vehicles, connected and automated or human-driven vehicles with navigation systems where human drivers follow instructions completely) utility-optimizing goal system's equilibrating processes in routing game among atomic selfish agents. Such can assist policymakers devising optimal operational planning countermeasures under both normal abnormal circumstances. To this end, we Markov (MRG) which each agent learns updates her own en-route path choice policy while interacting others transportation networks. efficiently solve MRG, formulate it as multi-agent reinforcement (MARL) devise mean field deep Q (MF-MA-DQL) approach captures competition The linkage between classical DUE our proposed is discussed. We show shown converge notion predictive dynamic user equilibrium (DUE) when traffic environments are simulated using loading (DNL). In other words, MRG depicts DUEs assuming perfect information deterministic propagated by DNL models. Four examples solved illustrate algorithm efficiency consistency equilibrium, on simple network without spillback, Ortuzar Willumsen (OW) Network, real-world near Columbia University's campus Manhattan New York City.